Skip to content

Two level quantization support - #76

Merged
gouyt13 merged 5 commits into
VectorDB-NTU:mainfrom
TamimEhsan:two-level-quant
Sep 1, 2026
Merged

Two level quantization support#76
gouyt13 merged 5 commits into
VectorDB-NTU:mainfrom
TamimEhsan:two-level-quant

Conversation

@TamimEhsan

@TamimEhsan TamimEhsan commented Aug 31, 2026

Copy link
Copy Markdown
Member

x+y quantization Phase 1: split a code into a base filter layer and an ex refine layer

This PR is a part of the roadmap discussed within the VectorDB-NTU team for adding support for generalizing 1+x split quantization format to y+x quantization

Adds the quantization primitives for x+y — one code of base_bits + ex_bits, stored once and split across two regions. The top base_bits form a cheap-to-scan filter layer with their own factors; the bottom ex_bits form the refine layer carrying the combined code's factors. The base bits are never stored twice: the refine step recovers the combined inner product from the filter step's via ip(base) · 2^ex_bits + ip(ex).

This generalises the existing 1+y split, where the base layer is always the single sign bit.

What's here

Layer Added
data_layout.hpp BaseDataMap / ConstBaseDataMap — base code plus its three factors
rabitq_impl.hpp xy_bits: code_factors, split_code_with_factor, validate_bit_size; total_bits::combined_code
rabitq.hpp quantize_xy_single
estimator.hpp xy_single_base_dist (filter), xy_single_full_dist (one-shot both layers)
query.hpp SplitSingleQuery takes base_bits, defaulting to 1
tests 8 new cases

Nothing is wired to a consumer yet — no batch variant, no IVF or HNSW path.
This is the primitive layer only.

Backward compatibility

SplitSingleQuery's new base_bits parameter is trailing and defaults to 1, so every existing call site is untouched. At base_bits == 1, kbxsumq() computes −(2^(1+ex_bits)−1)/2, bit-for-bit what it computed before.

Refactors to existing code

total_bits::combined_code factors out the sign-bit-over-magnitude construction that rabitq_scalar_impl and rabitq_full_impl both assembled inline; rabitq_scalar_impl now delegates to it, dropping a dim-sized buffer and a pass. rabitq_full_impl can't yet — ex_bits_code_with_factor produces the magnitude and the factors together.

Next Phase

  • Add a u8×s8 base kernel (VNNI) and a quantized-query variant for base bits distance estimation

Issues

  • Truncation penalty. The base code is the top x bits of a code whose rescale factor was optimised for x+y bits, so it is generally weaker than a standalone x-bit code: zero at base_bits == 1 (the sign bit does not depend on the scale factor), ~11–15% higher RMSE at 2–3 bits, ~3% at 5. The intended trade for not storing the base bits twice and for keeping the boosting identity exact, which independently-scaled layers would break.

Verification

Clean build, 41 tests, check-format and check-tidy all clean. Beyond the suite: combined_code was checked against the scalar loop it replaces over 46,080 comparisons spanning total_bits 1–9 with exact zeros seeded. No mismatches, every code inside [0, 2^total_bits).

METRIC_IP is untestedcode_factors has two metric branches and only L2 is exercised.

@gouyt13
gouyt13 merged commit 73226bf into VectorDB-NTU:main Sep 1, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants